[ISSUE #10446] Support batch deletion of topics and subscription groups in broker#10448
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds broker/admin support for batch deletion of topics and subscription groups via new remoting request codes and request bodies, plus client APIs and broker-side handling.
Changes:
- Introduce new request bodies and request codes for batch-delete operations.
- Implement broker processor logic to delete topic/group lists (including dedup + optional cleanup behaviors).
- Add persistence helpers for batch config deletion and tests covering the new admin paths.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| remoting/src/main/java/org/apache/rocketmq/remoting/protocol/body/DeleteTopicListRequestBody.java | Adds request body for batch topic deletion. |
| remoting/src/main/java/org/apache/rocketmq/remoting/protocol/body/DeleteSubscriptionGroupListRequestBody.java | Adds request body for batch subscription-group deletion (with cleanOffset flag). |
| remoting/src/main/java/org/apache/rocketmq/remoting/protocol/RequestCode.java | Adds new request codes for batch delete operations. |
| client/src/main/java/org/apache/rocketmq/client/impl/MQClientAPIImpl.java | Adds client APIs to invoke the new batch delete broker requests. |
| broker/src/test/java/org/apache/rocketmq/broker/processor/AdminBrokerProcessorTest.java | Adds unit tests for batch deletion request handling and persistence behavior. |
| broker/src/main/java/org/apache/rocketmq/broker/topic/TopicConfigManager.java | Adds batch topic-config deletion with single persist/update. |
| broker/src/main/java/org/apache/rocketmq/broker/subscription/SubscriptionGroupManager.java | Adds batch subscription-group deletion with single persist/update. |
| broker/src/main/java/org/apache/rocketmq/broker/processor/AdminBrokerProcessor.java | Routes new request codes and implements batch delete handlers; refactors pop retry topic collection. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
oss-sentinel-ai
left a comment
There was a problem hiding this comment.
Review by github-manager-bot
Summary
This PR addresses two related performance issues in AdminBrokerProcessor:
- Removes
synchronizedfromdeleteTopicsincetopicConfigTableis already aConcurrentHashMap - Adds batch deletion APIs (
DELETE_TOPIC_IN_BROKER_LISTandDELETE_SUBSCRIPTIONGROUP_LIST) to reduce network round-trips
Findings
- [Good] AdminBrokerProcessor.java:769 — Removed
synchronizedfromdeleteTopic. The underlyingtopicConfigTableis aConcurrentHashMap, so per-keyremoveis atomic. This eliminates unnecessary serialization of concurrent admin delete requests. - [Good] AdminBrokerProcessor.java — Added
deleteTopicList()anddeleteSubscriptionGroupList()handlers following the existing*_LISTconvention (consistent withUPDATE_AND_CREATE_TOPIC_LIST). - [Good] DeleteTopicListRequestBody.java / DeleteSubscriptionGroupListRequestBody.java — Clean request body classes extending
RemotingSerializable, following existing patterns. - [Good] MQClientAPIImpl.java:2202-2220 / 2281-2301 — Client-side batch deletion methods properly implemented with timeout handling.
- [Good] RequestCode.java — Added
DELETE_TOPIC_IN_BROKER_LISTandDELETE_SUBSCRIPTIONGROUP_LISTrequest codes. - [Good] TopicConfigManager.java / SubscriptionGroupManager.java — Added
deleteTopicList()anddeleteSubscriptionGroupList()batch methods. - [Good] AdminBrokerProcessorTest.java — Comprehensive test coverage including:
- Empty list validation (returns INVALID_PARAMETER)
- System topic rejection in batch
- Happy path with multiple topics
- Duplicate handling in batch
- Batch persist verification
Analysis
The implementation is well-structured:
- Backward compatibility: Existing single-item deletion APIs remain unchanged
- Consistent patterns: Follows the
*_LISTconvention established byCreateTopicListRequestBody - Proper validation: Empty lists and system topics are rejected
- Test coverage: Edge cases are well covered
Suggestions
- Consider adding a maximum batch size limit to prevent excessive memory usage or long-running operations.
- The batch persist test verifies deduplication — good attention to edge cases.
Verdict
✅ Approved — Well-designed performance optimization with proper validation and comprehensive test coverage.
Automated review by github-manager-bot
b3e851d to
75ae0ef
Compare
75ae0ef to
53cda76
Compare
2ac8039 to
9afd46f
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #10448 +/- ##
=============================================
- Coverage 48.13% 48.13% -0.01%
- Complexity 13379 13403 +24
=============================================
Files 1377 1379 +2
Lines 100730 100886 +156
Branches 13012 13041 +29
=============================================
+ Hits 48491 48559 +68
- Misses 46300 46368 +68
- Partials 5939 5959 +20 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Review by github-manager-bot
Summary
Adds batch deletion APIs for topics (DELETE_TOPIC_IN_BROKER_LIST) and subscription groups (DELETE_SUBSCRIPTIONGROUP_LIST), with proper per-resource authorization, input validation, and atomic-all-or-nothing semantics.
Findings
- [Info] Authorization — Correctly emits one
DELETEcontext per resource in the batch, preventing bypass when the body carries the target list instead of the header. This is a security-critical detail handled well. - [Info] Error handling — Returns a structured error response with the list of failed items and the first exception. Good for debugging partial failures.
- [Info] Dedup — Uses
LinkedHashSetto preserve insertion order while deduplicating. Reasonable choice. - [Info] Refactoring — Extracts
collectPopRetryTopics()from inline code indeleteTopic(). Good cleanup that avoids duplication. - [Info] Cleanup order — Deletes client config and subscription first, then topic config last. Allows retry after partial failure.
- [Warning]
AdminBrokerProcessor.deleteTopicInBrokerList()— TheRemotingCommand responseis created once at the top and reused. IfdeleteTopicInBrokerinternally sets response fields, ensure there's no state leakage between the success and error paths. - [Warning]
MQClientAPIImpl.deleteTopicInBrokerList()— The method creates the request body and sends it. Consider adding a timeout or batch size limit to prevent excessively large requests.
Suggestions
- Consider adding a max batch size constant (e.g., 100) to prevent abuse or accidental large requests.
- The
DeleteTopicListRequestBodyandDeleteSubscriptionGroupListRequestBodyclasses are clean. Consider adding@AllArgsConstructorfor convenience if not already present. - Cross-repo: If batch deletion is exposed in the dashboard or CLI, ensure those clients are updated to use the new APIs.
Overall: Well-structured feature with proper authorization and error handling. A few minor suggestions but nothing blocking.
Automated review by github-manager-bot
…n groups in broker - Add DELETE_TOPIC_IN_BROKER_LIST and DELETE_SUBSCRIPTIONGROUP_LIST request codes with corresponding RequestBody types - Implement deleteTopicList / deleteSubscriptionGroupList in AdminBrokerProcessor with deduplication and one-shot persist - Add deleteTopicConfigList / deleteSubscriptionGroupConfigList in TopicConfigManager / SubscriptionGroupManager (single persist per batch) - Add MQClientAPIImpl#deleteTopicInBrokerList / deleteSubscriptionGroupList client APIs - Cover changes with unit tests in AdminBrokerProcessorTest
9afd46f to
267bec0
Compare
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Review by github-manager-bot
Summary
Adds two new batch admin APIs — DELETE_TOPIC_IN_BROKER_LIST and DELETE_SUBSCRIPTIONGROUP_LIST — to AdminBrokerProcessor, following the existing *_LIST convention (e.g. UPDATE_AND_CREATE_TOPIC_LIST). Includes proper authorization context building for body-driven batch requests, new request body DTOs, and test coverage.
Findings
- [Info] Authorization handling for body-driven batch APIs is correctly implemented — the
DefaultAuthorizationContextBuilderdecodes the request body and emits oneDELETEcontext per item, preventing the annotation-based path from producing an empty context list (which would bypass permission checks). - [Info] Good test coverage in
DefaultAuthorizationContextBuilderTest— verifies blank entries are filtered, correct resource types, and per-item DELETE action. - [Warning] The batch deletion is not atomic — if one topic/group fails to delete (e.g. still in use), the remaining items in the list may still be deleted. Consider documenting this partial-failure behavior in the API response, or adding a
failFastflag. - [Info]
LinkedHashSetfor deduplication while preserving insertion order is a good choice for predictable behavior. - [Info] New
DeleteTopicListRequestBodyandDeleteSubscriptionGroupListRequestBodyfollow the existing pattern ofCreateTopicListRequestBody.
Suggestions
- Consider adding a response summary that reports per-item success/failure status (similar to how batch operations in other systems report partial results). This would help callers understand which items succeeded when partial failure occurs.
- Add a brief comment in
AdminBrokerProcessordocumenting the non-atomic semantics of batch deletion.
No blocking issues. Well-structured addition that follows existing conventions.
Automated review by github-manager-bot
Which Issue(s) This PR Fixes
Fixes #10446
Brief Description
Adds two new batch admin APIs to
AdminBrokerProcessorwhose names follow the existing*_LISTconvention (e.g.UPDATE_AND_CREATE_TOPIC_LIST,UPDATE_AND_CREATE_SUBSCRIPTIONGROUP_LIST):DELETE_TOPIC_IN_BROKER_LISTDELETE_SUBSCRIPTIONGROUP_LISTBulk metadata cleanup (e.g. tenant decommissioning, retention purge, cluster migration) currently requires N round-trips to delete N items and triggers N
persist()calls on the broker config files. These new APIs allow doing the same in 1 RPC + 1 persist + 1messageStore.deleteTopics(Set)call.Changes
RequestCode: addDELETE_TOPIC_IN_BROKER_LISTandDELETE_SUBSCRIPTIONGROUP_LIST.DeleteTopicListRequestBody { List<String> topicList }DeleteSubscriptionGroupListRequestBody { List<String> groupNameList; boolean cleanOffset }TopicConfigManager#deleteTopicConfigList(List<String>)andSubscriptionGroupManager#deleteSubscriptionGroupConfigList(List<String>): aggregatepersist()once per batch.AdminBrokerProcessor:collectPopRetryTopics(...)shared helper used by both single and batch paths.deleteTopicList(...)anddeleteSubscriptionGroupList(...)(bothsynchronized, consistent with existing single APIs) with input dedup (preserving order), system-topic / blank validation, one-shot persist, and one-shotmessageStore.deleteTopics(...)call.MQClientAPIImpl: adddeleteTopicInBrokerList(...)anddeleteSubscriptionGroupList(...)client helpers; both use an explicit null check (instead ofassert) so callers get a deterministicMQClientException(SYSTEM_ERROR, ...)wheninvokeSyncreturns null.Compatibility
DELETE_TOPIC_IN_BROKER/DELETE_SUBSCRIPTIONGROUPrequest codes and methods are unchanged; existing clients are not affected.*_LISTrequest codes are additive; older brokers will fall through togetUnknownCmdResponse, so older clients/brokers continue to work as today.How Did You Test This Change?
New unit tests added in
AdminBrokerProcessorTest:testDeleteTopicListInBroker— covers empty input, system-topic rejection, and the success path.testDeleteTopicListBatchPersist— verifies that:deleteTopicConfigListis invoked once (sopersist()runs once for the whole batch),deleteTopicConfigis not called on the batch path,messageStore.deleteTopics(...)is invoked once with the batched set,testDeleteSubscriptionGroupList— covers empty input and the success path withcleanOffset=true.Local verification: